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Student evaluation of instruction has been a much discussed issue resulting 
in diverse and often contradictory findings. Several issues seem to comprise 
the majority of concern vjith student ratings; among these are: the reliability 
of such ratings, the validity of the ratings, the use of the ratings and the 
effect of student ratings on teacher behavior change. Studies have been pub- 
lished which point to students* ability to reliably rate instructors (e.g. 
Costin, 1968) and other studies verify the validity of these ratings (e.g. 
McKcachie, 1969). The more common proposed uses for student ratings include 
providing course end feedback for instructors, evaluating teaching competence 
for promotion purposes and salary adjustments and, providing the student body 
with information for selecting courses (or more properly, instructors). The 
final issue, the effect of student ratings on instructor behavior, (i.e., im- 
proving the quality of teaching) has not received a great deal of attention. 
What evidence does exist is contradictory and covers the total spectrum of help- 
ful to harmful. Tuckman and Oliver (1968) found feedback from student evalua- 
tions helpful in producing student achievement gains whan feedback was given to 
high school instructors on a mid-semester student evaluation. Miller (1971), 
hov;evcr, found no difference in student ratings of instruction betvr?een a mid- 
semester and end-of-semester evaluation when freshmen teaching assistants were 
given feedback at the mid-semester. Finally Oles and Lencoski (1973) found that 
graduate teaching faculty did change their self-ratings as a consequence of 
receiving student evaluations but that some of the instructors made changes in 
'*a direction exactly opposite of what would have been expected from the student 
feedback information." (Oles and Lencoski, 1973) 
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Considering the importance student evaluations are presently being given 
it is puzzling that fev/ investigations have been conducted on the effect of 
student evaluation feedback on instructor teaching behavior. However, this 
research gap is not too surprising for several reasons. Generally, instructors 
do not appear to place a great deal of faith in course-end ratings, this apparent 
lack of faith mitigates against their use in a positive manner to promote instruc- 
tional changes. Also, traditionally, the teaching function has been viewed as 
the individual perogative of each instructor. The notion of academic freedom 
often precludes criticism of instructional procedures and professional ethics 
often dictate that such criticism should not be publically voiced. Perhaps, the 
most compelling reason behind the lack of research concerning teacher change as a 
result of student evaluation lies in the nature of the evaluation instruments 
used for student ratings. The type of student evaluations generally used pro- 
vide almost no information v;ith enough utility to suggest teacher change. 

The procedure usually follov;ed in gathering student evaluation of instruc- 
tion is to give students a standard or quasi-standard questionnaire at or near 
the completion of a course. This questionnaire usually contains items such as 
"This course was poorly organized." (Finkbeiner, et. al. , 1973), "Now that you've 
taken this course, v/ould you recommend it to a friend v;ho did not have to take 
it?" (Hartley and Hogan, 1972) and "Achievement of course objectives was helped 
by class activities." (College of Education Student Evaluation Form, 1973). 
Students are asked to rate the instructor on each item, using a scale, usually 
from 1 to 5. Instructor feedback consists of an average score for each item. 

There are three major difficulties with this approach. First, course-end 
evaluations contribute very little to the improvement in quality of either 
teaching or learning because of the timing of the infomation gained. The 
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course is over when the evaluation data is received. Second, the type of 
information gained is ordinarily so general and non-specific that it provides 
no reasonable specific goals for instructor behavior change. The third major 
problem lies in the nature of v7hat is evalaated. As Hartley and Hogan (1972) 
noted, most course evaluations do not mention the students* self-development 
as a result of the course, thus, ignoring student estimates of outcome or 
personal growth. 

Many of the problems resulting from the use of traditional forms of student 
evaluation of instruction may be dealt V7ith if the distinction between formative 
and summative evaluation is employed in seeking a solution. Bloom, Hastings 
and Madaus (1972) distinguished between formative and summative evaluation based 
upon the purposs:, portion of the course covered (time) and level of generalization 
to be included in the evaluation. Formative evaluation is defined as being 
™ade ^'...during a course, when (presumably) changes can be made in the trans- 
actions of subsequent instruction on the basis of current attainment (Bloom, 
Hastings and Madaus, 1972, p. 262).'* Summative evaluation is defined by Bloom, 
ct. al. (1972) as '...used to designate student assessment at the end of a course 
or topic unit, that is, \7hen no subsequent changes in treatment for that learning 
will be made (p. 262)." While this distinction has not apparently been applied 
to student evaluation of instruction, it appears to hold promise in solving some 
of the problems noted by others with student evaluation of instruction. This 
paper reports a formative system for student evaluation of university instruction, 
the analysis of the data and uses of the data to improve instruction. The for- 
mative evaluation procedure reported in this paper included two dimensions often 
included in student evaluations: quality of instruction and student feeling of 
personal gain. 
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Procedures 

Subjects and Setting 

Sixty students enrolled in a teacher education program served as subjects 
while taking a required course in educational psychology. The class met thirty 
times for fifty minutes during the spring quarter. On the first day of class 
the students were given the follov7ing materials and instructions: 

1. A course proclamation v;hich outlined the general objectives for the 
course, the class procedures and routines to be followed and the 
criteria to be used in grading. 

2. A class daily record sheet (see figure 1) on which students recorded 
formative evaluations of the instructor and themselves and the number 
of daily behavioral objectives required and completed on a day to day 
basis. 

3. An explanation (see figure 2) detailing the use of the class daily 
record sheet and formative evaluation procedures. 

4. A standard size file folder in vJhich all evidence of completion of 
daily objectives was kept as well as the class daily record sheet. 

Students v;ero divided into groups of five to seven members. A group folder 
vas distributed to each group for assignments that were to be completed by the 
groups as a whole. The students were requested to put all the folders of stu- 
dents in the group together and vjith the group folder and to put a rubber band 
around all the folders. This procedure facilitated the daily distribution and 
collection of student folders. All folders were checked following every class; 
this allowed the students* daily evaluations of instructional activities to be 
reviewed. 

Formative Evaluation 

Formative evaluation was carried out quickly and efficiently through the 
use of a five point scale similar to the type scale often used with course 
evaluation (see figure 2). Students were given several verbal examples of 
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possible class experiences for rating the class on a daily basis and asked to 
use what they considered their ''average" experience as a reference point for 
the rating scale. Two categories were rated each day; the student's evaluation 
of the instructional presentation and the student's evaluation of his personal 
gain from that day*s presentation. Students v/ere asked to become familiar v7ith 
the rating scale and V7ere reminded to complete the evaluations during the last 
minute of the first three classes. Follov7ing this, no reminders were issued and 
the procedure generally appeared to take no longer than 10 to 30 seconds for 
any student once the routine v;as established. 

At the completion of the course students were asked to total the number of 
objectives completed and required. The evaluations under self and instructor 
V7ere also totaled and averaged yielding an overall course evaluation score. 

Pvcsults and Discussion 

Several factors were considered in evaluating the results of the formative 
evaluation data. First, reliability results indicated that students were able 
to rate on a day to day basis V7ith a high degree of reliability (KR2Q = .882). 
In addition, the correlation between student evaluations of the instructor and 
their evaluations of personal value of the day's activities was .745. These 
procedures were carried out follov7ing the completion of the course. While the 
overall ratings by students V7ere quite high (average personal evaluation, 1.81, 
average instructor evaluation, 1.92) and indicated a high level of satisfaction 
by the students, this data yielded no significant information relative to a 
particular lesson. llov7ever, using the reliability data, it V7as discovered one 
day in particular did not contribute to high reliability. A review of the instruc- 
tor's comments which were recorded following each day's lesson indicated he felt 
the lesson for this day to be in need of revision (e.g., "poor objectives," 
"^'^r lecture presentation"). In this case the reliability data substantiated 
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the instructor's feelings about that day's cl:.ss. This indicated a need for 
revision of this lesson prior to rc-presentation. 

Perhaps of even more importance v;as the role the daily evaluations played 
in solving instructional problems with individual students and particular con- 
cepts during the course. A review of the daily ratings by the instructor on a 
class to class basis revealed that two students V7ere consistently rating the 
classes as being of little value to them and of lov; instructional quality. These 
students were given the opportunity to meet course objectives in a manner more 
consistent with their personal learning styles V7hich resulted in increased 
feelings by these students regarding the value and quality of the course. In 
addition, when there appeared to be a great deal of variation in student ratings 
for any single day, this was interpreted as indicating a lesson which lacked 
clarity. Hence, this served as a cue to the instructor for some needed remedial 
work to be offered regarding the concepts, skills or values taught for that day. 

Perhaps the major reason previous use of student evaluation of instruction 
has not resulted in instructor change (e.g. the same instructors seem to get low 
ratings quarter after quarter) i.5 that these course evaluations have not been 
designed to facilitate instructor change. Rather, most of these instruments 
appear to be designed to give a global overviev; of hovj an instructor functions. 
This type of sutnmative evaluation is useful vjhen administrative decisions re- 
garding salary adjustment are to be made or a student v;ishes to know who is 
considered a "good teacher." But, x^hen the goal of the instrument is to foster 
improved instruction, evaluation after the fact in tonns of generalities is not 
helpful because instruction consists of specific behaviors* 

I'Jhatever classroom interaction either helped or hindered the classroom 
learning situation can seldom bo appropriately manipulated vjith suiranative 
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evaluation only. Verbal and nonverbal teaching habits (eye contact verrus 
looking out the window or at the ceiling, throat clearing, OK's, flat expression 
less speaking, unthinking animation, poor teaching materials, etc.) are unlikely 
to change as a result of the global non-helpful statements such as '^instructor ' s 
personality v/as a negative factor in this course ' usually found on summative 
course evaluations. Even v/hen factors are identified to explain the dimension- 
ality of an instrument (e.g. Finkbcinerj et. al. , 1973) it is doubtful that 
scores reported on any dimension such as an instructor characteristics factor 
\7ill provide information specific enough to suggest some positive instructor 
action. Formative evaluation carried out on a class session by class session 
basis has the potential advantage of behavioral specificity. This specificity 
can facilitate changes in instructional programs because of the immediacy and 
specific nature of the feedback received from students. With formative student 
evaluation, student feedback can be directed at specific teaching techniques 
and specific lessons ns well as include student evaluation of personal gain 
(outcome) for each lesson. In this manner, the relative contribution of each 
instructional period to an overall evaluation of instruction may be assessed. 

It is as unrealistic to expect students to be uniform as it is to expect 
teachers not to vary. Teaching styles differ greatly and so do learning styles. 
Higher education has, to an unacceptable degree, ignored individual student 
differences. The major mode of instruction is the lecture; however, for many 
students this is not the optimal learning style though it probably serves v;ell 
for the vast majority. Most students who have different learning stylos have 
been removed from the educational system by the time they reach college. Still, 
a fev; do remain vjho v;ill profit greatly from a different instructional mode. 
Individuals who are not or who feel they are not responding satisfactorily to 
the dominant mode of instruction may be identified through their daily ratings. 
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As more and more pressure is put on public institutions of post-secondary 
education to accept a broader range of students, and to some extent guarantee 
these students success, it v/ill undoubtedly become more important to have effec- 
tive ways of identifying individual students who are not responding v/ell. In 
order to be useful and helpful to the student, this must be done early and 
continually in the instructional program. 

Some light may be shed on two controversial issues regarding student rating 
data from the results of the reported formative evaluation. First, the issue 
raised by Hartley and Hogan (1972) that ''we should be at least equally interested 
in the students' judgment of their ov7n self-development as affected by the 
course'* (p. 249) as in the students' subjective judgment of an instructor's 
behavior may be superfluous. The high correlation betv7een student daily ratings 
of the instructor and student evaluation of personal gain suggests either may 
serve as an indicator of the other. Hov;ever, this may be true only when the 
ratings are made on a day to day basis. Second, Rodin and Rodin (1972) indicated 
that students fail to recognize good teaching when they are exposed to it and 
KoGoff (1971) has expressed doubt vjhether good teaching can be judged at all. 
VJliile the data do not provide a conclusive ansv/er as to v/hether students can 
identify good teaching, there is evidence to indicate they can. The one day 
V7hich was discovered to be a low contributor to the reliability coefficient 
\jas also felt to be a poor day by the instructor, supposedly an expert in 
designing and evaluating instructional programs. This data at least provides 
some notion of validity, even if only based on the average perception by the 
class and the perception of the instructor. In any event, where such agreement 
exists regarding the inef fectivenes.<3 of a particular lesson, there is compelling 
reason for changes to be made in the instructional program for that day. 



- 9 - 



Fcv7 people are naturally good teachers; students often appear to feel that 
universities get less than their share of the fev7 v/ho are naturally endov7cd. 
Even those who do posscr quality instructional skills cannot expect these 
skills to be maximally effective for all students. If teachers are \7illing 
to seek out the students with v;hom they are ineffective and to also neet the 
needs of these individuals, the probability of significant learning occurring 
\7ill increase greatly. : The process of quality instruction is indicated as the 
probability of learning increases. The formative student instructional evalua* 
tion system described here is designed to enhance this process. 
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Figure 2. Instructions given to students explaining the use of the 
Class Record Sheet and the Formative evaluation procedures a 



Every day you are present in class you should complete your Class Record 
Sheet. This should remain current at all times. The Class Record Sheet is to 
be used to record daily progress in attempting and completing the daily objectives 
of the course. Each week you v7ill be given a schedule of upcoming topics to be 
covered during the V7eek. 

The following procedures are to be used in recording your progress on the 
Class Record Sheet. 

1. Write in the appropriate date in the space provided. 

2. (a) The number of objectives assigned for that day and (b) the number 
that you have completed. You must complete 90% of the assigned objectives 
to be eligible to receive a grade in this course. 

3. Under the heading evaluation record (a) your personal evaluation of the 
day's activities and (b) your evaluation of your instructor for that day. 

a) Personal evaluation of the day's activities. 
Use the following scales: 

1 = 1 feel this xcss an excellent day. I "learned" material 

which seems to be highly relevant to good teaching. I vjas 
very involved. 

2 « I feel this V7as a very good day. I ''learned" material 

which appears to be relevant to good teaching. I was 
involved most of the time. 

3=1 feel this was a good day. I 'learned" seemingly 
relevant material. I v7as involved a fair amount of 
time. 

4=1 feel this was a poor day. I "learned" some relevant 
material but not much. I v;as not involved to any great 
extent. 

5 = 1 feel this V7as a horrible day. I "learned* little if 
any material relevant to good teaching. I was almost 
totally uninvolved. 

This scale should be used V7ith your past academic experience as 
your reference point. That is, you should consider your acquisition 
of knowledge and skill in this class in relation to other classes you 
have had. Therefore, in relation to other classes in your past ex- 
perience, you may feel this class for the particular day is superior; 
if this vjere the case you would rate the class that day a one. On 
the other hand, you may feel a particular class is really rotten; you 
v7ould rate the class as a five for that day. 
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Under the heading Instructor you should record your evaluation 
of youx: instructor for that day. This rating should be made on 
the basis of the clarity and rationale of the class presentations, 
£ nd the quality of the instruction program you are presented. 
The follovjing scale should be used: 

1 = Excellent presentation, clear, concise; excellent skills 
programs 



2 =5 Very good presentation and/or skills exercise. 

Generally very clear and understandable. 

3 = Good presentation and/or skills exercise. Fairly 

clear and understandable. 



4 = Poor presentation and/or skills exercises. Unclear and only 

comprehensible with great effort. 

5 = Horrible presentation and/or skills exercises. Obscure, 

obviously irrelevant, pointless, a total waste of time. 



Finally, in the space provided at the bottom of the Class Record 
Sheet, make any comments you feel particularly relevant for that 
day*s class. You may not want to include a comment each day but 
when you do \jrite a comment please date it. You may make any comment 
you wish about the quality of the instruction and it would be most 
helpful if those comments pointed out specific particulary positive 
or negative instructor behaviors vjhich contributed to the quality 
of the day's class. Examples of such comments might be: '*You 
talked too much in a monotone voice. ' or **Your use of examples was 
very poor and confusing." or 'Your use of humor during the lecture 
was excellent and maintained interest." or ''The personal attention 
you give each student helps in learning the material/* 
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ABSTRACT 

A procedure for using the concept of formative evaluation in student 
evaluation of instruction is presented. Sixty undergraduate students in an 
educational psychology course completed daily evaluations of every instruc- 
tional program. The procedure v/as carried out quickly and efficiently through 
the use of a 5 point scale and a standard reporting format. The results v/erc 
reviev/ed daily by the instructor. Results indicated the procedure is both 
reliable and valid. In addition, the procedure appears to offer data useful 
in revising instructional procedures during the conduct of the course. Thus, 
the improvement of instruction is greatly facilitated due to the specificity 
of the data produced. Other implications for improving the quality of 
instruction are discussed relative to meeting individual student needs. 
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